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Abstract — In this paper, we consider uplink transmissions 
involving multiple users communicating with a base station over 
a fading channel. We assume that the base station does not 
coordinate the transmissions of the users and hence the users 
employ random access communication. The situation is modeled 
as a non-cooperative repeated game with incomplete information. 
Each user attempts to minimize its long term power consumption 
subject to a minimum rate requirement. We propose a two 
timescale stochastic gradient algorithm (TTSGA) for tuning the 
users' transmission probabilities. The algorithm includes a 'wa- 
terfllling threshold update mechanism' that ensures that the rate 
constraints are satisfied. We prove that under the algorithm, the 
users' transmission probabilities converge to a Nash equilibrium. 
Moreover, we also prove that the rate constraints are satisfied; 
this is also demonstrated using simulation studies. 

I. Introduction 

Wireless networks have witnessed large scale proliferation 
in the recent years. Apart from voice applications, data ap- 
plications such as World Wide Web (WWW), email etc. are 
also extremely popular with the users. Different applications 
have different Quality of Service (QoS) requirements from 
the network in order to perform satisfactorily. These QoS 
requirements can be in terms of parameters such as the 
delivered rate, delay or delay jitter. In this paper, we consider 
providing QoS (average rate) guarantees to web applications 
such as WWW operating over a single cell wireless network. 
This entails addressing the following important issues: 

• Medium access control (MAC): Multiple users need to 
access the common wireless channel simultaneously in 
order to communicate with a common receiver such as 
a base station or an access point. The access mechanism 
must be so designed that it satisfies the QoS requirements 
of user applications. 

• Challenges offered by the Wireless Medium: Wireless 
channel is characterized by decay of signal strength due 
to distance (path loss), obstructions due to objects such 
as buildings and hills (shadowing), and constructive and 
destructive interference caused by copies of the same 
signal received over multiple paths (multipath fading), 
possibly with time varying path lengths, resulting in a 
time varying channel condition [1]. These phenomena 
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distort the signal in an unpredictable manner and can 
cause packet errors at the receiver. The time varying 
wireless channel poses significant challenges for design 
of efficient and reliable communication systems. 

Various MAC protocols have been proposed in the literature 
that attempt to satisfy different application requirements based 
on their packet arrival characteristics and QoS attributes. These 
protocols can be classified into following two types: 

• Fixed resource allocation protocols: These protocols as- 
sign fixed amount of resources to the users by means 
of orthogonal or near orthogonal channels. They require 
a central scheduling entity (like a base station) that per- 
forms the channel allocation task. Examples include Time 
Division Multiple Access (TDMA), Frequency Division 
Multiple Access (FDMA), Code Division Multiple Ac- 
cess (CDMA) and Orthogonal Frequency Division Mul- 
tiple Access (OFDMA). These have been implemented in 
cellular systems [1]. 

• Random access protocols: In these protocols, users access 
the channel randomly. Users vary their channel access 
probabilities or access times based on limited feedback 
from the channel. Since the users take transmission 
decision based on local information available with them, 
these protocols are suited for distributed implementation. 
Random access protocols have been implemented in 
cellular systems, satellite communication systems and 
multitap bus among others. These have been well studied 
for the past several decades. [2], [3] provide excellent 
textbook treatment of random access protocols. 

Recent research on designing efficient transmission strate- 
gies over fading wireless channels has revealed a lot of inter- 
esting insights which suggest that fading can be considered as 
an opportunity instead of it being treated as an adversary. Users 
located at diverse locations are likely to perceive diverse chan- 
nel conditions (multiuser diversity). When the number of users 
is large, the probability that a certain user perceives very good 
channel condition is close to one [4]. This multiuser diversity 
has been exploited for designing efficient cross layer schemes 
[5] that utilize this information from the physical layer in order 
to make efficient scheduling decisions at the MAC/network 
layer (e.g., scheduling the user perceiving the 'best' channel 
condition in each slot). Several studies have demonstrated that 
this strategy substantially improves performance in terms of 
throughput [4], [6]. 

In this paper, our objective is to design a random access 
scheme for providing average rate guarantees to the contending 
users. We consider the uplink scenario where multiple users 
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transmit to a single base station over the wireless channel. 
The users take advantage of the opportunities provided by the 
wireless channel by varying their transmission powers based 
on their channel condition ('channel aware' transmission). 
Since wireless devices are battery powered with limited battery 
energy, in order to conserve energy, the objective is to devise 
a random access and power control scheme wherein the long 
term average power consumption for each user is minimized 
subject to satisfying the average rate requirement of each user. 

There have been several attempts to design channel aware 
random access schemes in order to improve the performance 
of these schemes. Existing work on channel aware random 
access schemes can be classified into the following classes: 

• Signal processing and diversity techniques to correctly 
decode received packets as in [7], [8], [9], [10]. 

• Work that advocates the adaption of retransmission prob- 
abilities of users either through 'Splitting algorithms' that 
adapt the set of contending users based on feedback from 
the channel as in [11], [12] or through 'channel aware 
ALOHA' schemes as in [13], [14], [15], [16]. 

Our work falls under the latter category. 

A. Related Work 

In this section we review representative research that ad- 
vocates adapting retransmission probabilities of users under 
channel aware ALOHA schemes. 

In [13], [17], the authors attempt to exploit multiuser 
diversity in a distributed fashion with only local channel infor- 
mation, i.e., each user is aware of its own channel condition 
only. The authors propose a channel aware ALOHA protocol 
and provide a throughput analysis of the proposed protocol 
under an infinitely backlogged model. In [15], the authors 
consider symmetric as well as asymmetric fading. The authors 
propose a binary scheduling algorithm where users access the 
channel when the corresponding channel condition is above a 
certain threshold and prove that it maximizes sum throughput 
under symmetric fading. Moreover, for asymmetric fading, 
they prove that binary scheduling maximizes the sum of log 
of average throughput of the users and is fair in the long run. 
Furthermore, they also consider channels with memory and 
provide simple extensions of the binary scheduling algorithm. 
In [14], the authors study distributed schemes for exploiting 
multiuser diversity in the uplink (multipoint to point) context. 
They propose a channel aware ALOHA scheme where the 
transmission probability is a function of channel state infor- 
mation. They characterize the maximum stable throughput for 
such a system with both finite as well as infinite user models. 
In [16], the author defines an interference-dominating wireless 
network as the one in which a receiver could simultaneously 
receive a number of packets from a variety of transmitters, 
as long as the signal-to-interference-plus-noise ratio exceeds 
a predetermined threshold. The author proposes an analytical 
approach to derive the exact value of saturation throughput of 
slotted ALOHA in such an interference-dominating wireless 
ad-hoc network. 

In addition to the above, there has been a lot of interest 
in power control techniques over random access wireless 



networks. Game theory [18] serves as a useful tool for design- 
ing these power control schemes. Moreover, it also provides 
a useful framework for designing access control protocols 
with provision for information exchange (cooperative game) 
as well as no information exchange (non-cooperative game) 
between users. Game theoretic models have been extensively 
applied [19], [20], [21], [22]. The slotted ALOHA protocol 
has been modeled both as a non-cooperative game as well as 
a cooperative game. Various objectives like delay minimization 
[19], throughput maximization [19], [20], [21] have been 
considered. Power control coupled with retransmission control 
has been variously studied in [23], [24], [25]. In [26], the 
authors analyze the equilibrium points achieved by a non- 
cooperative group of users that have a certain QoS requirement 
and willingness to pay. The reader is referred to [24], [27], 
[28] for further information on applications of game theory 
for modeling the random access problem. 

The model considered in this paper is similar to that in [17]. 
However, in [17], the authors consider the throughput scaling 
under long term and short term power constraints, while we 
consider a different problem of minimizing the long term 
power expenditure of each user subject to satisfying the rate 
constraint of each user. The problem formulated in this paper 
is similar to that of a recent work in [29]. In [29] the authors 
assume that users transmit at constant power, while our for- 
mulation also includes power control. Moreover, while in [29], 
the authors' emphasis is on analysis of the Nash Equilibria of 
the game involving a group of non-cooperative users sharing 
a channel and desiring certain long term throughput, we adopt 
a prescriptive approach. We focus on algorithm design and 
show that the Two Timescale Stochastic Gradient Algorithm 
(TTSGA) suggested by us converges to the Nash equilibrium 
of the game under certain conditions. 

B. Our Contributions 

In this paper, our objective is to provide QoS (average rate) 
guarantees to users while taking advantage of the opportunities 
provided by the fading wireless channel. We assume that the 
distribution of the channel fading process is not known to 
the users. We assume that each user accesses the channel 
independently of others based on its Channel State Informa- 
tion (CSI) only. Moreover, the users are not aware of the 
rate requirements and the channel conditions of the other 
users. Furthermore, there is no mechanism for information 
exchange between the users. This situation is modeled as a 
constrained repeated non-cooperative game, where each user 
has an objective of minimizing the long term average power 
expenditure subject to achieving a certain long term average 
rate. The uses modulate their transmission rates based on the 
CSI fed by the base station. We propose an iterative primal- 
dual technique for tuning the transmission probabilities of the 
users and ensuring that the constraints are satisfied. The primal 
variable is the transmission probability while the dual is the 
waterfilling threshold that adjusts the average transmission 
power for ensuring that the rate constraint is satisfied. Our 
contributions can be summarized as follows: 

• We formulate the user problem where the objective is 
to minimize average power consumption subject to an 
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Fig. 1. Uplink transmission scenario 

average rate constraint as a constrained repeated non- 
cooperative game where each user has knowledge of its 
rate requirement and CSI only. 

• We propose a Two Timescale Stochastic Gradient Al- 
gorithm (TTSGA) for iteratively tuning the transmission 
probability. This algorithm iterates the transmission prob- 
ability in the direction of the gradient of the average 
power consumption. Moreover, the 'waterfilling thresh- 
old' is tuned on a faster timescale to ensure that the rate 
constraints are satisfied. 

• We prove that under TTSGA, the transmission probability 
iterates converge to a Nash equilibrium provided certain 
conditions are met. 

• The approach ensures that the rate constraints are met. 
This is proved analytically and validated through simula- 
tions. 

The rest of the paper is organized as follows. Section II 
provides details regarding the system model. We formulate 
the problem as a constrained repeated game in Section III and 
then motivate a solution within the random access framework. 
In Section IV, we motivate the solution strategy and propose 
TTSGA. In Section VI we analyze certain properties of the 
algorithm and prove that under the algorithm, the transmission 
strategies of users converge to a Nash equilibrium. Moreover 
at equilibrium, the rate constraints are satisfied. We present the 
simulation results in Section VII. In Section VIII, we discuss 
implementation aspects of the algorithm within IEEE 802.11 
framework. We conclude in Section IX. 

II. System Model 

We consider an uplink scenario similar to that in [17] as 
depicted in Figure 1 where N users communicate with a base 
station. We consider a time slotted system, i.e., time is divided 
into slots of equal duration normalized to unity. There can be 
multiple flows between a user and the base station. However, 
for the sake of notational simplicity, we assume that only one 
flow exists between a user and the base station. The analysis 
can be easily extended to more general cases. 

We assume that the system operates in a distributed fashion. 
Hence we assume that the base station does not coordinate the 
transmissions of the users. In each slot n, user i transmits 
with a certain probability 8 l n to the base station. If more 



than one user transmits in a slot, then all transmissions are 
unsuccessful, i.e., there is a collision at the base station. We 
assume that each user receives a (0,1, e) feedback in each 
slot, where denotes that there is no transmission in the slot, 
1 denotes successful transmission and e denotes collision or 
unsuccessful transmission. We assume that this feedback is 
immediate and error free. In practice, this information can be 
conveyed through acknowledgement messages sent by the base 
station over a feedback channel. 

We assume a wireless channel with block fading [30]. Under 
this model, if xli K the transmitted signal by user i in slot n, 
then the signal Y£ received by the base station in slot n can 
be expressed as: 

Y* = H t n7 & + Z n , (1) 

where Z n is complex Additive White Gaussian Noise (AWGN) 
at the base station. W n is the channel gain and we denote 
X % n = \H l n \ 2 G X as the channel state for user i in slot 
n. We assume that the users possess perfect knowledge of 
channel state X l n in each slot '. Moreover, the distribution of 
X % n is not known to user i. We discuss possible mechanisms 
for conveying information such as successful reception of a 
packet and CSI in Section VIII. 

All packets are assumed to be of equal size, say, £ bits. We 
assume a backlogged model, i.e., users always have packets to 
transmit. Let \] % n denote the number of packets that user % can 
transmit reliably to the base station in slot n. Since the slot 
duration is normalized to unity, \J % n can be considered to be the 
transmission rate in slot n. In practice, this can be determined 
based on the modulation or coding scheme employed at the 
physical layer. Let P i (X^ l ,U^ l ) denote the power consumed 
by user i while transmitting at rate U l n when the channel state 
perceived by the base station is X % n . 

III. Problem Formulation 

In this section, we formulate the problem as a repeated non- 
cooperative game. We begin by considering the ith user. The 
long term power consumption for user i can be expressed as: 




where 6 l n denotes the transmission probability for user i in slot 
n. Let (3 l n denote the probability of successful transmission for 
user i in slot n. The long term throughput or rate achieved by 
user i can be expressed as: 

1 M 

^ = liminf-^/3^. (3) 

M—>oo M — ' 
n=l 

The optimization problem for user user i can be expressed 

as: 

Minimize T l subject toW > p\ (4) 

where p 1 denotes the long term average rate requirement for 
user i. 

'in practice, the users perform channel estimation for downlink transmis- 
sions using the pilot symbols transmitted by the base station. In a Time 
Division Duplex (TDD) system because of symmetry, these estimates can 
be used for uplink transmissions as well. 
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We are interested in determining an 'equilibrium' or 'steady 
state' transmission probability for each user (say 9 1 '* for user 
i) and a channel state dependent power control policy such that 
if a user transmits at this probability with the power prescribed 
by the power control policy, the average power is minimized 
and the rate constraint is satisfied. Note that the above problem 
can be viewed as a rate constrained random access problem 
where each user i solves a similar optimization problem. 

Remark 1: Let V l ei denote the average power expenditure 
for user i when the transmission probability is fixed at 9 l . It 
consists of two parts: 'useful' power, i.e., the power consumed 
in successful transmissions and 'wasted' power, i.e., the power 
wasted in collisions. Let and P % £i denote the 'useful' 
power and 'wasted' power respectively when the success and 
collision probabilities are (3 l and e l respectively. The problem 
for each user is to determine an equilibrium transmission 
probability 8 h * such that P\ it , = V l (i%t , + P^,, is minimum. 

The problems being solved by N users are not independent. 
Transmission probability of one user (say user i) impacts 
the collision/success probabilities of all the other users. This 
affects their transmission probability which in turn impacts 
the transmission probability of user i. Thus, this is a game 
situation [18]. Each user attempts to minimize its own dis- 
utility (power) subject to its rate requirement. We assume that 
a user is not aware of the rate requirement of the other users. 
Moreover, since there is no provision for information exchange 
between the users, the CSI is also localized at the users, i.e., a 
user is not aware of the CSI of other users. Furthermore, since 
the users are only provided with a (0, l,e) feedback by the 
base station, the users cannot fully observe the actions taken 
by the other users. We view the situation as a repeated non- 
cooperative game with incomplete information. The solution 
concept that we target is that of the Nash equilibrium. In this 
case, at equilibrium, each user's transmission strategy in the 
long run can be viewed as a 'best response' to the long term 
transmission strategies of the rest of the users. 

IV. Single User Scenario 

Before proceeding with the multiuser scenario, we analyze 
the single user scenario that provides us with key insights that 
aid in designing an efficient multiuser solution. 



A. Single User Scenario 

Consider a single user scenario as depicted in Figure 2. We 
assume that the user user is split into two virtual entities: a 
transmitter and a scheduler. The user transmits over a block 
fading channel in a time slotted system. The scheduler has 
an objective of minimizing the long term average power 
expenditure subject to an average rate constraint (say p) 2 . To 
meet this objective, in each slot, the scheduler determines 
the transmission rate (say U n ) based on the channel state 
(say X n ) and directs the transmitter to transmit at that rate. 
However, with a certain probability f3, the transmitter is unable 

2 Since we deal with the single user scenario, in this section, we omit the 
superscript from the notation for notational simplicity. 
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Fig. 2. Single user scenario 



to proceed with the transmission. In this case, the long term 
throughput achieved by the user can be expressed as: 



1 M 

t/ = liminf — V pu n . 



(5) 



n=l 



Moreover, the long term power consumption can be expressed 
as: 

M 

P = lim_sup — J2/3P(X n ,U n ). 



M^oo 



71=1 



The problem can be precisely expressed as: 

Minimize V subject to U > p. 



(6) 



Note that the present problem is a generalization of dual of 
the problem considered in [31]. The analysis for determining 
the optimal transmission policy proceeds on similar lines as in 
[31]. The constrained problem in (6) can be converted into an 
unconstrained problem using the Lagrangian approach [32]. 
The unconstrained problem can be expressed as: 



Minimize V - X(U - p), 



(7) 



where A > is referred to as the Lagrange Multiplier (LM). 

It can be verified that for the present problem, the optimal 
power allocation for a channel state X n — x can be expressed 
as: 

(8) 



/>„(*) = (A* - -f ), 

where Nq is the power spectral density of the AWGN at the 
base station. The rate U n under channel state X n can then be 
determined based on this power allocation. Note that P w (-) 
depends on the optimal LM A* which in turn is a function 
of the transmission probability [3. This optimal LM A*, which 
we also refer to as the 'waterfilling threshold', however, needs 
to be determined. 

Remark 2: Note that in the single user scenario, the trans- 
mitter transmits with a probability (i and remains idle with 
probability 1 — (3. The slots where the transmitter does not 
transmit can be considered to be equivalent to the slots 
where the channel state is extremely poor. For such slots, 
the Goldsmith-Varaiya (G-V) waterfilling power allocation 
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scheme [31] does not transmit. It is easy to see that the 
G-V scheme is optimal for this case also, albeit with an 
appropriately scaled LM or the 'waterfilling threshold' as 
compared to the case where the transmitter transmits in each 
slot. Effectively, as (3 decreases, the scheduler perceives a 
progressively poorer channel. This results in a correspondingly 
higher value for the optimal LM A* resulting in higher 
power allocation in each channel state for satisfying the rate 
constraint. 

Remark 3: It can be argued that for the single user scenario, 
in the optimal solution, the constraint is met with equality. 
We know that power is an increasing convex function of 
the transmission rate. Now, suppose that the constraint is 
not met with equality, then a further reduction in power 
consumption can be achieved by transmitting at lower rates 
and then meeting the constraint with equality. This implies 
that if the constraint is not met with equality, the solution is 
not optimal. Hence, in the optimal solution, the constraint is 
met with equality. 

Remark 4: Note that for the single user problem considered 
in (6), the waterfilling power allocation scheme (8) is optimal. 
For different values of transmission probability, the optimal 
LM A* takes different values. Hence in the multiuser situation, 
we employ (8) for each user user i for determining the trans- 
mission power at a given channel state. The task that remains is 
to determine the equilibrium transmission probability 1 '* (and 
thereby the success probability and the corresponding 

constraint satisfying LM A* * for each user. 

V. Iterative Approach for Determining the 
Equilibrium Transmission Strategy in the 
Multiuser Scenario 

In the previous section, we discussed the single user sce- 
nario. In this section, we focus our attention back to the 
multiuser scenario. In determining the equilibrium transmis- 
sion probability within the multiuser setting, a user attempts 
to address the following tradeoff: if it were to transmit with 
too high probability, there could be too many collisions and 
wastage of power, on the other hand if it were to transmit 
with too low probability, it would amount to transmitting at 
higher power in each channel state in order to satisfy the rate 
constraint. The users attempt to achieve a balance between 
these conflicting objectives for arriving at a solution. 

We now suggest a strategy for determining the equilibrium 
transmission probabilities. The essence of the solution strategy 
is the following: each user tunes its transmission probability 
iteratively so as to arrive at a Nash equilibrium. The users 
adapt their transmission probabilities in the direction of the 
gradient of average power expenditure. Moreover, the water- 
filling threshold is also iteratively tuned so as to ensure that the 
rate constraints are satisfied. Before providing details of the 
solution strategy, we convert the constrained problem in (4) 
into an unconstrained problem using the Lagrangian approach 
[32]. 

A. Lagrangian Approach 

Let A* be a Lagrange Multiplier (LM). The unconstrained 
problem (corresponding to the problem in (4)) can be stated 



as: 



Minimize C\9\ X 1 ) = P l - X i {U i - p l ). 



(9) 



The objective is to determine the saddle point of the La- 
grangian C l (9\X l ), i.e., to determine 1 '* and A*'* such that 
the following saddle point optimality conditions are satisfied: 



(10) 



Note that the LM also acts as the waterfilling threshold. Hence, 
determining the equilibrium LM also results in determining the 
equilibrium waterfilling power allocation. 

B. Two Timescale Stochastic Gradient Algorithm (TTSGA) 

We now present a stochastic gradient algorithm for the prob- 
lem in (4) for a given user i. The essence of the algorithm is 
the following: in each slot a user i determines its transmission 
power P^(X^) based on its channel state X l n using (8). User 
i transmits with probability 9 % n that serves as an estimate of 
the equilibrium transmission probability 9 1 -* . User i tunes this 
estimate in the direction of the gradient of the Lagrangian in 
(9). In Section VI we prove that this algorithm leads to a Nash 
equilibrium. 

The Lagrangian in (9) can be expressed as: 

1 M 

C{9\X) = limsup-^(^(x;)-V(/3;(7;-p 1 )) 

M 



(ID 



where we refer to g(-, •,-,-,•) as the immediate cost function. 
Let V e *L'(-, •) and \7 y L l (-, •) denote the partial gradient of 
L l {-, ■) w.r.t 9 % and A 1 respectively. At the saddle point, 9 h * 
and A 4 '* satisfy the following conditions: 



V e *C{9\\ 1 ) 



W y C l (9\X) 



= o, 
= o, 



and the complementary slackness condition, 

A*'*(W* — p*) = 0. 



(12) 
(13) 



(14) 



Note that the Lagrangian in (11) is a time average of the 
immediate cost function that can not be determined a priori 
in a real time implementation setup. If the equilibrium LM 
A''* is known, we can use an iterative method that improves 
its estimate of only the equilibrium 9 l . Since the equilibrium 
LM A" is also not known, we resort to a primal-dual method 
that determines both 9 h * and A" iteratively [32]. In order 
to ensure convergence of 9 % and X 1 iterates to the equilibrium 
9 1 '* and A 4 '*, the iterations proceed at different timescales, i.e., 
the 9 l and A* values are updated at different rates [33]. We 
iterate 9 % on a slower timescale, and A 1 on a faster timescale. 
This implies that 9 l is maintained constant for a large number 
of A* iterations. More specifically, as viewed from the A* 
iteration, the 9 % iterates appear to be almost constant while 
as viewed from the 9 % iteration, the X % iterates appear to be 
converged to the optimal value for the current value of 9 l . 
The can be done in two ways. The first way is to physically 
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separate the timescales by updating the LM in each time slot 
and updating the probability after a large number of time slots. 
The second way is by updating both quantities in each time 
slot, but carefully selecting the update sequences employed in 
the iteration. It can be shown that this has the same effect as 
that of the physical separation of time scales [33]. Let {a n } 
and {c„} be two positive sequences that have the following 
properties: 

a„^0,c„^0; ^a„ = oo, ^c„ = oo; 

n n 

^(a„) 2 < oo; ^(c„) 2 < oo. (15) 

k n 

Fix % = 6. The equilibrium LM for this transmission proba- 
bility that ensures that the complementary slackness condition 
(14) is satisfied can be determined using the following iteration 
carried out in each slot: 



A^+i = K - an(J l n K - p l ), X n > Vn, 



(16) 



where J % n is an indicator variable that is set to 1 if user i 
transmission is successful in slot n. (16) forms the 'waterfilling 
threshold update mechanism'. A 4 is the waterfilling threshold 
for user i, increasing this threshold results in an increase in 
power consumption, while decreasing this threshold results in 
a decrease in the average power consumption. 

We now describe an approach for iterating the transmission 
probability 6 l in the direction of the gradient of the average 
power expenditure. This approach also involves separating the 
Q % and A 4 update timescales by carefully selecting the update 
sequences. It involves transmitting with probability 9 % + S in 
a odd numbered slots and with probability & l — S in even 
numbered slots at powers recommended by the waterfilling 
power allocation scheme (8). Let (2n— 1) and (2n) refer to odd 
and even numbered slots respectively. Hence a user i transmits 
with probability (#2„_i+£),c> > 0, 5 « 1, and (0* 2n _ 1 -5) in 
odd and even numbered slots respectively. Based on the finite 
differences method [34], the gradient of the average power 
expenditure is determined and the transmission probability is 
updated in the direction of this gradient in odd numbered slots. 
This update equation can be expressed as: 



*2n+l - *1 



~>2n-l _ c 2n-l 



v. 



2 n-l 



v. 



2n 
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(17) 



where V n is an estimate of the average power expenditure 
for user i in slot n. Note that we do not have access to this 
estimate of the average power expenditure. In order to address 
this issue, we carry out simultaneous averaging of the power 
consumed. V n is thus computed using the following recursive 
equation: 



k +1 = k+w^to, 



(18) 



where b n is an update sequence that has the same properties 
as those of a n and c„ in (15). 

We update V % on a faster timescale as compared to the 
probability update timescale. This is done by imposing ad- 
ditional properties on update sequences b n and c„ explained 
below. This ensures that as viewed from the average power 



update timescale, the transmission probability appears to be 
almost constant; the physical interpretation being that one is 
computing the average power expenditure for a certain large 
time interval with fixed value of the transmission probability. 
Note that in (18), 6 l can be considered to be the current 
quasi-static value of the transmission probability as seen from 
the power update timescale. LM is updated at the fastest 
timescale because it determines the waterfilling threshold. 
This guarantees that the scheme uses the correct waterfilling 
threshold and hence the transmission power P^(-) is the 
correct power which in turn leads to correct average power 
values V 1 . 

The different timescales specified above can be realized by 
imposing the following additional requirements on the update 
sequences {a n }, {&„}, {c„} [33]: 



0; 



0. 



(19) 



Practically, these timescales can be realized by having, e.g., 

(16), (18) and (17) and form the Two Timescale Stochastic 
Gradient Algorithm (TTSGA). In a nutshell, TTSGA con- 
sists of updating three quantities: LM that determines the 
waterfilling threshold, the average power consumption and 
transmission probability. These three quantities are updated on 
different timescales. In Section VI-A, we show that if each user 
implements TTSGA then the transmission probability vector 
converges to a Nash equilibrium. Moreover, the LMs converge 
to values such that the rate constraints are satisfied. 

VI. Analysis of TTSGA 

In this section, we comment on several properties of TTSGA 
(17), (18) and (16) such as convergence and fairness. We begin 
with convergence analysis. 

A. Convergence Analysis 

In this section, we prove that: 

• The probability iterates under TTSGA converge to a Nash 
equilibrium under certain conditions. 

• Rate constraints are satisfied. 

Let 6 n = [9 n , . . . , 9„] denote the vector of transmission 
probabilities of the users. For a fixed transmission probability 
vector 9 = [6 1 , . . . , 6 N \, let (3 l denote the probability of 
successful transmission for user i. It can be expressed as: 



p l = e l \\[\-0 1 ). 



(20) 



In this case, average power and LM update equations can 
be expressed as: 



v n+1 = r n + b n e*pux n ), 



(21) 



and 



K+i = K - a n (P l U n - p*), K > Vn. (22) 
(22) can be expressed as: 
K+i - K-a^FUi-FUi+pui-?), \ n > Vi, (23) 
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where \] % n is the running average of the transmission rate and 
^(U^ — Un) f° rms a martingale difference sequence. Note that 
the LM determines the waterfilling threshold and hence the 
power with which a user transmits in each channel state which 
in turn determines the average rate that the user achieves. 
Hence {/* can be expressed as: 

U i n = F i {\ i n ), (24) 

for some continuously differentiable function F l (-). Using 
(24), (23) can be expressed as: 

K +1 = K-anipui-FF^x^+FF^xiyp 1 ), y n > o v», 

(25) 

Consider a 'fluid approximation' of (23) with the interpretation 
that we consider smaller and smaller slot lengths, so that 
power and transmission probability are interpreted as 'per 
unit time' quantities instead of 'per slot' quantities. Under 
this approximation, (22) can be considered to be a noisy 
discretization of the following ordinary differential equation 
(o.d.e.): 

\ i (t) = -((3 i F i (\ i (t))-p i ). (26) 

The set of equilibria of the o.d.e. in (26) H = {A*'* : 
/3 i F i (\ i '*) = p 1 }. The stability of the o.d.e. (26) allows 
us to comment on the convergence of the iterates in (22). 
Using Theorem 2, Chapter 2 (p. 15) of [35] we can claim the 
following: 

Lemma 1: For a fixed vector, the LM iterates in (22) 
converge to the set H of equilibria of (26). 
Note that convergence of the LM iterates directly implies that 
the rate constraint is satisfied with equality. 

Remark 5: The situation at each user i with fixed 6 has sim- 
ilarities with the single user scenario; the probability of success 
[3 l can be treated as an analogue of the transmission probability 
(3 in the single user scenario. In the multiuser scenario, the 
probability of success determines the waterfilling threshold 
A\ higher the probability of success, lower the threshold and 
hence lower the power consumed. However, the parameter that 
the user controls is the transmission probability. Increasing the 
transmission probability increases the probability of success 
but also increases the probability of collision. The user seeks 
a balance between these two. 

Consider the transmission probability update equation: 



?2n+l = Tl 



hn-l ~ c 2n-l 



■pi _ -pi 

26 



(27) 



For the sake of analysis, let us drop the projection operator. We 
will comment on the projection operation later in this section. 

Now, consider a fluid approximation of (27) expressed as 
the following o.d.e.: 



6\t) = -VfliP'W, 



(28) 



where Vgi denotes the gradient w.r.t. l . The system wide 
vector o.d.e. can be expressed as: 



e(t) = -VeP(t), 



(29) 



Where P is the power vector [P 1 , ■ ■ ■ ,V N ]. It is clear that 
VeP(-) acts as a Lyapunov function (See [35] Chapter 10, 



(10.2.1)), i.e., 



— P(t) = -||V^P(i)|| 2 < 



dt 



(30) 



Let G = {6 : VqP = 0} denote the set of local minima of 
P, i.e., the equilibrium points for this o.d.e. 

We now prove that the Hessian H of P at = 6* 
under certain conditions is positive definite. This implies that 
9* e G denotes a stable equilibrium point of the o.d.e. 
(29) [36]. In fact, the equilibrium is a Nash equilibrium 
[36] (See also Proposition 2 of [26]). In order to prove the 
positive definiteness of H at 6 = 6* , we show that it is 
strictly diagonally dominant which is a sufficient condition for 
positive definiteness ([37], Section 4.2.1). Next, we evaluate 
the components of the Hessian H. 

Recall that VL denotes the average power consumed by user 
i when the transmission probability is 6 l . It consists of two 
components: V 1 ^ which corresponds to the power consumed 
in successful transmissions and V\i which corresponds to the 
power wasted in collisions. Let p(x l ) denote that the user i 
has channel condition x l . The average throughput for user i 
with a success probability of (5 % can be expressed as: 



/3^p(^)log 2 (l + 



iV 



), 



(31) 



where P^ix 1 ) is the transmission power in channel state 
x % when the success probability is (3 l . From Remark 3, the 
constraint is satisfied with equality. Hence, using (14) and (31), 
we can claim that: 



/?'^>(Oiog 2 (i+ e y N J ) 



E^) lo g2(i 



) = 



(32) 



Let p l (x' 1 ) denote the rate at which user i transmits in 
channel state x % when /3* = 1. From (32), 

log2(1 + 3i^ ) = ^-) 



N 
No 



P^)=-^2^-l). 



(33) 



It can be seen that P^x 1 ) is a convex decreasing function of 
P\ Now, 

V% = (*') = ^ P ( S , )f(2^ - 1). (34) 

Hence V 1 ^ is also a convex decreasing function of f3 l . V % ei can 
be expressed as: 



/ p. 



(35) 



Lemma 2: The following condition is a sufficient condition 
for the Hessian H to be positive definite: 

--Y 1 



0* 



(1 - k >*) 



> 0. 



(36) 
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Proof: Note that, 
d{Vh) 



E^)( 



t ^ iVo^(x')ln(2) ^g!ff) 



(fli)2(ei)2 

where e 4 = — Moreover for fc 7^ i, 



Furthermore, 



iV 2 0< (e l 6>* + \n{2)p l (x 1 )) 



(! _ ky^ e iy e i x i 



(37) 



)• (38) 



5 2 (7>L) 



9(^) 2 



ix (N \n{2)p l {x l )2 ^ {2t l 6 l + ln(2)p i (x i )) 
x*((9*) 4 (e*) 3 



)■ 

(39) 



Finally, 

1 0(0*0*) 1 ^(owy 



A^ ln(2)p l (x l )2 ^ (2e 4 ^ +ln(2)p l (cc i )) 



a; i(6»i)3( e i)3( 1 



(40) 



From [37], Section 3.4.10, the following two conditions 
are sufficient for strict diagonal dominance of H at = #*; 



first: 



second: 



d(0>) 2 



> Vi which is easily verified from (39); and 



d 2 (^) , V | d 2 (n0 | 
9(^) 2 1 0(0*0*) 1 



> 0. 



(41) 



0=0* 



With little algebraic manipulation on (39) and (40), it can be 
shown that this condition is equivalent to: 



1-y 1 

Qi,* L^i (\ _ Qk,* 
k^i 



> 0. 



(42) 



Note that since the % iterates are updated on the slower 
timescale, these iterates see converged values of LMs at each 
update instant. Finally, when the iterates converge to the 
equilibrium, the corresponding LM values while ensuring that 
rate constraints are satisfied, also determine the correct long 
term power consumption. 

Remark 6: One way to ensure the condition in (42) is to 
enforce a limit on the maximum probability with which a user 
can transmit in a slot. We already have this mechanism in place 
through the limit u> on transmission probability. The second 
term in (42) takes its maximum value when 9^ = cj, Vfc, while 
the first term takes its minimum value when 0*>* = lu. For the 
minimum value of the LHS to be greater than we require 
that: 

1 N ~ 1 >0^ U <L (43) 



1 



N 



Condition (43) forces the transmission probability iterates in 
the interval [0, ^). If there exists an equilibrium point in the 
interval [0, ^), the iterates converge to such an equilibrium. 

B. Equilibrium as a Best Response 

Nash equilibrium embodies the notion of best response 
offered by a player to the strategies of the other players. 
Unilateral deviation from the equilibrium does not result 
in an increase in the utility for any player. In the present 
case also the transmission probability is a best response to 
the transmission probabilities of the other players. Unilateral 
deviation from the equilibrium transmission probability does 
not result in a decrease in the average power consumption 
for a player. This is because decreasing the transmission 
probability by, say, user i from its equilibrium value decreases 
both - its success probability (i l and collision probability e\ 
This increases V 1 ^ and reduces V\ % ; the net effect being that 
there is an increase in the overall power consumption. On 
the other hand, increasing the transmission probability from 
its equilibrium value increases fi l but also increases e\ This 
reduces V l ^i but increases V\ % \ the net effect being that there 
is again an increase in the overall power consumption. 



Positive definiteness of Hessian enables us to claim the 
following: 

Theorem 1: If there is a stable equilibrium point 6* e G, 
such that (42) is satisfied, then for any initial transmission 
probabilities the dynamics 6{t) in (29) converge to 6* 
asymptotically. 

Note that so far, we have studied the convergence of 
the o.d.e. (29). However, we are really concerned about the 
convergence of the primal TTSGA iterates in (27). Theorem 
2, Chapter 2 (p. 15) of [35] allows us to claim the convergence 
of these iterates: 

Theorem 2: If there is a stable equilibrium point 0* e G of 
the o.d.e. (29), then for any initial transmission probabilities 
the 6 n iterates in (27) converge to 9*. 

Finally, using Theorem 2, Chapter 6 (p. 66) of [35] we claim 
that the coupled iterates converge, i.e., 

Theorem 3: The coupled iterates (A\0*) in (16) and (27) 
converge to their respective equilibrium values. 



C. Multiuser Penalty 

Let V\ ((3 l = 1) denote the average consumed by a user 
i in the single user scenario with successful transmission in 
every slot. In the multiuser scenario, (3 l < 1. Since the user 
is able to successfully communicate only during a fraction 
of the slots, the rate constraint appears to be appropriately 
scaled, resulting in a corresponding scaling of the LM or the 
waterfilling threshold. This results in a corresponding increase 
in the power consumption. This is the penalty that the user 
pays in operating in a multiuser environment. We term this 
penalty as the multiuser penalty. Note that larger the number 
of users, potentially larger is the penalty paid by the user for 
obtaining a certain throughput. 

D. Fairness 

A user does not have an incentive for arbitrarily increas- 
ing the transmission probability, it can be increased only 
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when the reduction in power consumption due to increase 
in success probability outweighs the corresponding wastage 
due to increased probability of collision. The useful power 
consumption depends on two factors: the rate constraint and 
average channel condition. A user having a high rate constraint 
would require high rates in each channel state as compared to 
a user having a lower rate constraint but with same channel 
statistics. A user with higher rate constraint would therefore 
require more 'useful' power. Increasing the transmission rate 
increases the success probability and reduces the 'useful' 
power requirement. Therefore, such a user would have a 
higher transmission probability but this also results in higher 
collision rate and higher 'wasted' power. Similar arguments 
can be made for users with same rate requirement and different 
average channel conditions. This discussion implies that users 
having higher rate requirement or poorer channel consume 
more power thus ensuring fairness. 

In the next section, we simulate TTSGA in a discrete 
event simulator. Our objective is to demonstrate that TTSGA 
satisfies the rate constraints through simulation studies. 

VII. Experimental Evaluation 

In this section, we simulate a single cell wireless system 
where N users communicate with the base station on the 
uplink. The user applications require average rate guarantees. 
Packets are generated at the application layer and are possibly 
of variable sizes. At the MAC layer, we assume that each 
MAC fragment is of constant length equal to I = 2000 bits. 
We assume that the system has a bandwidth W of 10 MHz. 
Each user transmits at a constant power of 1 Watt. We assume 
uj = 0.1. We simulate a Rayleigh channel for each user. For 
a Rayleigh model, channel state X 1 is an exponentially dis- 
tributed random variable with mean ct and probability density 
function expressed as fx{x) = jj2 cxp (^j§2^j , x > 0. We 
discretize the channel into eight equal probability bins, with 
the boundaries specified by { (-oo, -8.47 dB), [-8.47 dB, 
-5.41 dB), [-5.41 dB, -3.28 dB), [-3.28 dB, -1.59 dB), 
[-1.59 dB, -0.08 dB), [-0.08 dB, 1.42 dB), [1.42 dB, 3.18 
dB), [3.18 dB, oo ) }. For each bin, we associate a channel 
state and the state space X = { -13 dB, -8.47 dB, -5.41 
dB, -3.28 dB, -1.59 dB, -0.08 dB, 1.42 dB, 3.18 dB}. This 
discretization of the state space of X 1 has been justified in 
[38]. For the sake of simplicity, we assume that the rate of 
transmission for user i in slot n can be determined using the 
following capacity relation: 



lP n = W x log 2 (l 



P Y l 

N W 



)■ 



(44) 



We consider a system with N = 20 user destination pairs. 
We divide the user destination pairs into 2 groups (Group 1 
and Group 2) of 10 pairs each. We consider two scenarios: 
rate variation and channel variation. We present the results 
after averaging over 20 simulation runs each consisting of 
simulating the algorithm for 100, 000 slots. 

Scenario 1: Rate Variation: In this scenario, we vary the 
average rate constraints for the pairs in Group 2 in successive 
experiments while keeping the average rate constraints for 



Power variation with rate 




Rate constraint for users in Group 2 



Fig. 3. Power expended for various average rate constraints 



Transmission probability variation with rate 



50 60 70 80 00 1 

Rate constraint ot users in Group 2 



Fig. 4. Transmission probability (TP) for various average rate constraints 



the pairs in Group 1 constant in all the experiments. For 
Group 2, the rate constraints are varied as 50-110 Kbps 
in steps of 10 Kbps in successive experiments, while the 
rate constraints for Group 1 are kept at 50 Kbps in all 
experiments. The mean channel state a for all the destinations 
is kept at -3.28 dB (0.4698) for all the experiments. In 
each slot, we generate the channel state using the exponential 
distribution with mean a and subsequently discretize it using 
the probability bins as mentioned above. Each user makes 
the scheduling decision using its transmission probability l n 
in each slot. Based on the feedback received from the base 
station, it then determines the new value of the transmission 
probability. We select two users, at random from Group 1 and 
Group 2. For these pairs, we determine the power consumed, 
the stable transmission probabilities and rate achieved within 
each experiment and plot these in Figures 3, 4 and 5 respec- 
tively. It can be seen from Figure 5 that the rate constraint 
satisfied. Moreover, from Figures 3, 4 it can be seen that as 
the rate constraint is increased, the power expended and the 
transmission probability increase. 

Scenario 2: Channel Variation: In this scenario, we vary 
the average channel state for the pairs in Group 2 in suc- 
cessive experiments while keeping the average channel state 
for the pairs in Group 1 constant in all experiments. For the 
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Fig. 5. Throughput for various average rate constraints 



Fig. 7. Transmission probability (TP) for various average channel states 
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Fig. 6. Power expended for various average channel states 



Fig. 8. Throughput for various average channel states 



pairs in Group 2, the average channel state a is varied as 
0.05 (-13 dB), 0.1422 (-8.47 dB), 0.2877 (-5.41 dB), 
0.4698 (-3.28 dB), 0.6934 (-1.59 dB), 0.9817 (-0.08 dB), 
1.3867 (1.42 dB) in successive experiments, while the av- 
erage channel state for the pairs in Group 1 is kept at 
0.4698 (—3.28 dB) in all experiments. The average rate 
constraint is kept constant at 50 Kbps for all the pairs for 
all experiments. In each time slot, we generate the channel 
state using the exponential distribution with mean a and sub- 
sequently discretize it using the probability bins as mentioned 
above. We select two users at random from Group 1 and Group 
2. For these pairs, we determine the power consumer, the 
stable transmission probabilities and rate achieved within each 
experiment and plot these in Figures 6, 7 and 8 respectively. 8 
demonstrates that the rate constraints are satisfied. Moreover, it 
can be seen from Figures 6, 7 that as the average channel state 
improves, the transmission probability reduces thus reducing 
the average power expenditure. 

VIII. Practical Implementation 

In this section, we first describe in brief the operation of Car- 
rier Sense Multiple Access/Collision Avoidance (CSMA/CA) 
access control protocol employed in IEEE 802.11 [39]. We 
then describe a protocol for practically implementing TTSGA. 



This protocol is based on a modification of IEEE 802.11 
CSMA/CA access framework. We then discuss implementa- 
tion details relevant to TTSGA within this framework. 

A. IEEE 802.11 CSMA/CA Protocol 

In CSMA, a user that desires to transmit has to first sense 
the channel for a predetermined amount of time. If the channel 
is sensed 'idle' then the user proceeds with its transmission. 
If the channel is sensed 'busy' then the user has to defer its 
transmission for a random duration of time (backoff). This is 
done to reduce the collision probability. 

Wireless local area networks (LANs) have the 'hidden 
terminal' problem, wherein a transmitter Ti transmitting to 
a receiver R that is not in transmission range of another 
transmitter T2 does not know whether T2 is transmitting to R. 
This can lead to collisions at R. This hidden terminal problem 
is avoided through the Collision Avoidance (CA) mechanism 
of IEEE 802.11. The CA mechanism involves transmission of 
Request to Send (RTS) packet by transmitter Ti to receiver 
R (refer to Figure 9). The receiver, if idle, sends the Clear to 
Send (CTS) packet back to T\. This alerts all the transmitters 
in the range of T\ and R of the data transfer between T\ and 
R. The other transmitters then suspend their transmissions till 
the duration of the data transfer (Data portion in Figure 9). 
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Clear to send (CTS) Acknowledgement (ACK) 




Request to send (RTS) 



Fig. 9. Slot structure 

Transmission of the Acknowledgement (ACK) packet from R 
to Ti signals successful transfer. 

If T\ after transmitting RTS does not receive CTS for a 
certain period of time, it infers that it was involved in a 
collision. In order to avoid further collisions, it backs off or 
defers its transmission using an 'exponential backoff mecha- 
nism prescribed by the protocol. This mechanism consists of 
backing off randomly by selecting a backoff duration from a 
window [0, W] referred to as the contention window. Each 
time a transmitter is involved in a collision, it doubles its 
contention window and chooses a random backoff duration 
from that window. 

B. A Protocol for Practical Implementation of TTSGA 

We now describe a protocol for practically implementing 
TTSGA. This protocol replaces or supplements the exponential 
backoff mechanism of the IEEE CSMA/CA protocol. A user 
first contends for the channel by transmitting a RTS packet 
based on the current transmission probability computed by 
TTSGA. The RTS packet is sent at the most robust rate to 
guard against channel fading. CSI can be estimated by the 
users based on a pilot transmitted by the access point. In a 
TDD system, the same CSI can be used by the users for uplink 
transmissions. CSI can be conveyed to the base station using 
ranging messages. There might be multiple users contending 
for the channel by transmitting RTS packets to the access 
point. If there is no collision between these RTS packets, the 
access point responds with a CTS packet. This packet also 
contains information regarding rate at which the user should 
transmit based on the CSI. If the user receives the CTS packet, 
it utilizes the transmission rate information in the CTS packet 
and transmits accordingly. If the access point receives the data 
transmitted by the user devoid of errors, it sends an ACK 
packet back to the user. The user then enters the contention 
mode again by transmitting the RTS packet with a recomputed 
transmission probability based on TTSGA. 

When the user receives a CTS packet corresponding to a 
different user, it suspends its transmission till it receives an 
ACK packet corresponding to that data transfer. This signals 
the completion of the transmission between the base station 



and the associated user. Reception of the appropriate CTS 
packet allows the user to determine whether it was involved in 
a collision or not. If an appropriate CTS packet is received, the 
user determines the number of packets that can be transmitted 
in the Data part of the slot and transmits those packets. It 
then recomputes the transmission probability based on whether 
the transmission was successful or not. Thereafter, the user 
contends for the channel by transmitting a RTS packet with 
the newly computed transmission probability. 

C. Practical Implementation of TTSGA 

Based on online primal-dual computations in (16), (18) and 
(17), and the protocol suggested in previous sub-section, user i 
implements the access control scheme. The user is aware of the 
value of channel state X 1 in each slot through the CTS packet. 
The number of packets to be transmitted is then determined 
using (8). The user i transmits with probability 6 % + S in 
odd numbered attempts and with probability 9 % — S in even 
numbered attempts. The transmission probability is adjusted 
in odd numbered attempts. If a transmission is successful, 
U l packets are received at the base station and an acknowl- 
edgement packet is received and the LM X 1 is appropriately 
updated. The algorithm thus continues. The complete scheme 
is explained in Algorithm 1. 



1: Initialize the LM Aq <— 0, 9q <— 8q, n <— 1, channel state 

xi <- o. 

2: while TRUE do 

3: Transmit RTS packet with probability % n , 

4: if CTS received then 

5: Use CSI X l n in CTS packet to determine f7*. 

6: Transmit U % n packets. 

7: else 

8: lP n «- 0. 

9: end if 

10: if ACK received then 

11: J« - 1. 

12: else 

13: J* <- 0. 

14: end if 

15: Update the LM using (16). 

16: Update transmission probability 8 l n using (17). 

17: n <— n + 1. 

18: end while 

Algorithm 1: Two Timescale Stochastic Gradient Algorithm 
(TTSGA) 

Remark 7: Note that in this paper, we assume that the users 
transmit at a 'reliable' rate. Under this assumption, there are 
no transmission errors and users would always receive an ACK 
packet. The purpose of including ACK in the protocol is to 
indicate to the other users about the end of a transmission 
so that they can start contending for channel access. The 
case where transmission errors do occur and ACK reception 
(non-reception) is an indication of successful (unsuccessful) 
transmission forms an interesting future work. 
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IX. Conclusions 

In this paper, we have considered uplink transmissions 
in a single cell multiuser wireless system. The base station 
does not coordinate the transmission of the users, hence the 
users employ random access communication. In each slot, 
the users obtain a (0,1, e) feedback from the base station. 
The users have an objective of minimizing their long term 
power consumption while achieving certain long term average 
rates. We have modeled the situation as a constrained repeated 
non-cooperative game where users have knowledge of their 
utility function only. We have proposed a two timescale 
stochastic gradient algorithm (TTSGA) at the users in order to 
tune their transmission probabilities. The algorithm includes a 
'waterfilling threshold update mechanism' that appropriately 
tunes the waterfilling threshold for each user and ensures 
that the rate constraints are satisfied. We have proved that 
under the algorithm the transmission strategies converge to a 
Nash equilibrium and that the rate constraints are satisfied. 
Moreover, our simulation studies have also demonstrated that 
the rate constraints are satisfied. 
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